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Abstract 



The Revised Learning Process Questionnaire has been part of the development of a conceptual 
understanding of how students learn and what motivates them to engage in particular tasks. We 
obtained responses from 329 student volunteers at a mid-sized public university in the southeast 
United States. While looking at the psychometric properties of this questionnaire in a different 
educational context from which the instrument had been originally validated was one purpose of 
the study, the main thrust of this research was to use Profile Analysis via Multidimensional 
Scaling (PAMS) to improve the diagnostic functionality of the instrument as well as further 
explore the validity of the questionnaire. We found that interpreting the latent structure in terms 
of the dimensions of Strategy and Motive as opposed to the factors of Deep and Surface 
approaches to be more appropriate for diagnostic use. We also found that PAMS has the inherent 
ability to assess an individual’s fit within the model, thereby acting as a measure of self-report 
credibility. Both the Strategy and Motive dimensions were found to have ecological validity 
through analyzing its relationship to academic performance. 




Profile Analysis via Multidimensional Scaling for the Revised 
Two-Factor Learning Process Questionnaire 
Learning and study strategies have been an interest in higher education for decades. From 
the earliest studies done by Entwistle and Ramsden (1982), looking at the self-perceptions of 
learning and motivation in the academic environment, researchers have been interested in 
recording how students study. In recent years, instruments have been developed to record and 
analyze the phenomenon, to better understand how people study and what factors can be 
attributed to success or failure in the academic setting. Kember, Biggs, and Leung (2004) and 
Everson, Weinstein, and Laitusis (2000) are examples of those who have endeavored to evaluate 
student learning goals, and in addition suggest the realignment of many general education and 
core curricula across many higher education institutions to help better cater to learning styles and 
study strategies of college students. Although some research has concentrated on more 
economic concerns, such as retention of students who struggle academically (Kember & Leung, 
2009), other studies have been focused on the development of instruments to delve more into the 
construct of learning and study strategies (Kaplan, 2008). 

The development of learning and studying as a construct has taken various paths. In the 
early 1970’s some of the earlier research was geared toward evaluating how students learn, and 
within what context does that learning occur. Marton and Saljo (1976a, 1976b), one of the first 
studies to attempt to describe levels of processing and examine qualitative differences in how 
students learn, has been the foundation for many studies attempting to develop instruments to 
measure these constructs. Some of these researchers see learning to be more related to the 
impetus or motivation of the learner, whether it be intrinsic or extrinsic (Grant & Dweck, 2001), 
while others recognize the motivational aspects but combine it with cognitive factors (Lufi, 




Parish-Plass, & Cohen, 2003; Scraw, 1994; Tobias & Everson, 1997) . Yet even combining these 
aspects, there are differing opinions of how they are manifested. For example, Weinstein, 

Palmer, and Schulte (2002) see the learning and study strategies to be a composite of constructs 
related to skill, will, and self-regulation, while Biggs, Kember, and Leung (2001), conceptualize 
learning as being composed of motive and strategy, which in either case is ether surface or deep 
depending on the complexity of the particular attribute. 

For this paper, however, our interest was not to debate the difference between theories, or 
to champion one specific explanation, but to move forward in the evolution of the instrument 
itself. That is, as the course of the history of researching learning and study strategies, numerous 
studies have been conducted in order to determine a more perfect understanding of the 
representation of these ideas from the data collected. In all cases research designs have been 
constructed and analysis undertaken to better able to researcher to understand not only how the 
construct manifests itself, but also how the instrument that assesses the construct operates. For 
example, Entwistle and Ramsden (1982) looked at the data qualitatively to help first identify the 
rudimentary questions to be utilized on a survey instrument, in the earliest development of the 
Learning Process Questionnaire. Later, Kember et al. (2004) ran a variety of exploratory factor 
analyses to determine the latent structure of the constructs, and later, confirmatory factor 
analyses to see how well the instrument represents that construct. 

More specifically, these instruments have been evaluated and validated. Utilizing factor 
analytic procedures helped refine the constructs that emerged from the data collected. For 
example, for the Learning Process Questionnaire (LPQ) the conceptualization of Deep Approach 
and Surface Approach were realized. Biggs et al. (2001) went further to demonstrate that motives 
and strategies are found within these learning approaches by the representation of the factor 




structure. In the analysis of the LPQ and its subsequent revisions, Kember et al. (2004) were able 
to clearly show the factors found in the instrument, and how those factors related to one another. 

It is now imperative that we take yet another step forward, and that is to show how to 
apply the instrument when describing samples of people. In short, the construct has been 
confirmed, the instrument’s ability to detect that construct has been confirmed, and it is now time 
to apply the instrument to help students. Factor analysis determines how the construct is 
manifested within the instrument (exploratory) and whether the instrument consistently 
demonstrates the existence of the factors for that construct (confirmatory), while PAMS looks at 
the instrument in terms of what it tells us about people. 

The current study was conducted with two major purposes. The first is to evaluate the 
psychometric properties of the Revised Two-Factor Learning Process Questionnaire, a later 
version of the LPQ (R-LPQ-2F; Kember et al., 2004). It is important to note that the sample used 
to confirm the two-factor version of the questionnaire was comprised of secondary students in 
Hong Kong, which required that the survey be translated into Chinese. As there may be some 
subtle issues in translation from English to Chinese, as well as cross-cultural differences in 
academic experiences, it is essential to determine if the Deep Approach and Surface Approach 
structure can be replicated with a more rural, American population. In addition, in order to use 
this survey for a sample of post- secondary students, it is important to replicate the findings with 
that age group. This included evaluating the ecological validity of the instrument by looking at 
relationships between the instrument factors and academic performance indicators such as GPA 
and student retention. In addition, this study engaged in the next level of progression in the 
analysis, by using Profile Analysis via Multidimensional Scaling (PAMS) in order to explore the 
characteristics of respondents of the R-LPQ-2F. 




As PAMS can be used to develop profiles for individual test takers, it can be used as a 
means for improving the diagnostic functionality of the instrument. The two-factor structure, 
identified by Kember et al. (2004), is indeed useful for describing the construct of the learning 
process and identifying how the instrument operates in terms of Deep Approach and Surface 
Approach. However, being able to use the instrument to interpret the data affords a richer 
utilization of the instrument, both diagnostically and prescriptively. For example, which students 
are more likely to be retained, maintain high GPA’s, or finish within 4 years? Describing student 
in terms of the dimensions of strategy and motive will allow us to identify “those with him 
motive low strategy” or “high strategy use, high motivation” and be better prepared to associate 
that with their academic successes or failures. 

PAMS is a method that can be used to explore individual profiles of test takers for 
assessments with multiple subtests. Profiles such as these can be instrumental in looking at 
strengths and weaknesses of test takers within the subtests as well as affording the ability to 
make an evaluation that can be used to remediate problem behaviors (Kim, Davison, & Frisby, 
2007). Also, PAMS allows a researcher to identify students who develop in an idiographic 
manner or are not explained by the model for whatever reason, whereas this capability is not 
inherent in factor analysis. 

Method 

Participants 

We distributed a web-based version of the R-LPQ-2F to students at a mid-sized public 
university in the southeast United States. A total of 329 volunteers (227 women and 102 men) 
responded. Ages ranged from 17 years old to 64 years old (M = 27). Approximately 89.4% were 
white and 31.3% were graduate students. Since the R-LPQ-2F and its psychometric properties 




were originally produced by a large sample of secondary school students in Hong Kong, our 
sample is important in determining whether those psychometric properties are consistent in a 
university setting and across cultures. 

Materials 

The R-LPQ-2F consists of 22 items which can be summarized into two broad factors of 
Deep Approach and Surface Approach. Within Deep Approach, there are subscales for Deep 
Motive and Deep Strategy, and, within Surface Approach, there are subscales for Surface Motive 
and Surface Strategy. Further, these subscales contain two subcomponents each. Items were 
answered on a 5-point Likert-type scale ranging from 1 ( this item is never or only rarely true of 
me) to 5 ( this item is always or almost always true of me). Standard R-LPQ-2F instructions were 
presented to participants. Appendices A and B contain the questionnaire and scales, respectively 
(Kember et al., 2004). 

Results 

Construct Validity 

First, we conducted a confirmatory factor analysis using EQS (Version 6.1) to determine 
if the full hierarchical model (see Figure 1) found by Kember et al. (2004) has good construct 
validity with our sample. The model was fitted from the covariance matrix. The variances for the 
Deep Approach and Surface Approach constructs were fixed to one, which is less restrictive than 
constraining factor pattern coefficients (MacCallum, 1995). This presumes that the variables are 
independently estimated for different groups and that the same model fits different groups 
(Thompson, 2004). Hu and Bentler (1999) recommended that a Comparative Fit Index (CFI) 
value greater than or equal to .96 in combination with a Standardized Root Mean Squared 
Residual (SRMR) value less than .09 would minimize the errors of rejecting a model when it is 




true and accepting a model when it is false. As a result, the CFI and SRMR were used to analyze 
model fit. 

The CFI fell below the cut-off (CFI = 0.810), but the SRMR was right at the cut-off 
(SRMR = 0.092). These values were not as good as those found by Kember et al. (2004), where 
the CFI was 0.967 and the SRMR was 0.036, but are still good. It is possible that our lower fit 
indices are due to a smaller sample size (. N = 329 compared to N = 801). It is important to note 
that all of the paths were statistically significant at the a = .05 level, indicating that all items and 
lower-order factors have a significant and useful contribution to the model. Also, we found a 
modest negative correlation between Deep Approach and Surface Approach whereas Kember et 
al. (2004) found a modest positive correlation. Intuitively, it makes more sense for this 
correlation to be negative because a positive correlation implies that a student who utilizes more 
surface strategies and motives would be more likely to also be utilizing more deep strategies and 
motives. The standardized solution for the hierarchical model is shown in Figure 1. 

Next, multidimensional scaling (MDS), conducted in SAS (Version 9.1.3), was used to 
further explore the latent variable model of the R-LPQ-2F. Dimensions in MDS are similar to 
factors in factor analysis. In fact, if the data satisfy a simple structure factor model then those 
factors can be seen within the MDS solution even though the MDS dimensions and factors do 
not need to neatly map onto each other (Davison & Skay, 1991). The variable space in factor 
analysis is representative of the underlying relationships of a set of attributes with respect to a 
sample of individuals. MDS is different from factor analysis in that dimensions are interpreted as 
characteristics of the objects to which the individual pays attention when he/she makes 
judgments. In other words, MDS focuses on item differences more than individual differences. 




MDS uses proximities, or measures of how similar or dissimilar different objects are, to 
derive a geometric configuration of points representing the “hidden structure” of the data with 
each point representing an object (Kruskal & Wish, 1978). These proximities were calculated 
using the city-block, or Minkowski- 1, metric from observed student responses of the R-LPQ-2F. 
This metric is suitable for describing certain types of psychological data because it emphasizes 
psychological judgmental processes (Weinberg, 1991). 

These proximities are used to derive dimensions, or coordinate axes, representing 
underlying characteristics of the objects under study. The goal is to obtain the lowest 
dimensionality that best explains the underlying structure of the data (MacCallum, 1974). 
Choosing the dimensionality is usually done through the use of goodness-of-fit statistics, 
Krustal’s stress formula 1 (STRESS 1) being a common statistic. This stress value is the square 
root of a normalized “residual sum of squares,” and indicates the level of fit for a specific 
dimensionality (Kruskal & Wish, 1978). STRESS 1 values closer to 0 represent better 
configurations. Stress values partially depend on the number of objects and the dimensionality, 
so for the most accurate interpretation of stress the number of objects should be large compared 
to the number of dimensions. One rule of thumb is that the number of objects should be more 
than four times the number of dimensions (Kruskal & Wish, 1978). Since the R-LPQ-2F contains 



22 items, the number of dimensions should be no more than five. 




Figure 1 . Hierarchical Model for Motive and Strategy Scales in R-LPQ-2F 





Figure 1. Hierarchical Model for Motive and Strategy Scales in R-LPQ-2F (N = 329). 











A plot of STRESS 1 versus dimensionality is useful in determining how many 
dimensions to retain and can be interpreted in a similar manner to the scree plot in an exploratory 
factor analysis. An “elbow” in the plot suggests that additional dimensions offer negligible 
improvement of fit, thereby giving an indication about the lowest dimensionality that best 
explains the underlying structure of the data (Weinberg, 1991). A solution should not be 
accepted if the STRESS 1 is above 0.10 (Kruskal & Wish, 1978; Manly, 2004). Also, increasing 
the number of dimensions is questionable once STRESS 1 is already less than 0.05 (Manly, 
2004). The plot of STRESS 1 versus dimensionality for our data shows an elbow at two 
dimensions (see Figure 2). This two dimension solution has a STRESS 1 value of 0.08. 



Figure 2. STRESS 1 Values Plotted as a Function of Dimensionality 




# Dimensions 

Figure 2. STRESS 1 values plotted for MDS solutions of different dimensionalities. An “elbow” is an indication 
about the true number of dimensions (Weinberg, 1991). This elbow should not be accepted if the STRESS 1 value is 
above 0.10 (Kruskal & Wish, 1978). This graph signifies that the two-dimension solution has the most accurate 
representation of the proximity measures with the lowest dimensionality. 

We scaled the two dimensions (i.e., normalized) to a root-mean-square value of 1 and 
adjusted the dimension coefficients to compensate (see Table 1). These dimension coefficients 
describe the relationships between the variables which represent characteristics of the individuals 



in our study (see Figure 3). The high points and low points are typically mirror images of each 




other and can be used to label the dimensions (Ding, 2001). The low end of Dimension 1 on this 



graph is represented by items 4, 8, 12, 16, and 22. The items are all Surface Strategy items in the 
hierarchical factor model. The high end of the dimension is represented by items 2, 10, 11, 14 
and 15. Items 2, 10, and 14 are all Deep Strategy items in the hierarchical factor model and items 
11 and 15 are Surface Motive items in the factor model. Based on the MDS solution, Dimension 
1 appears to represent Strategy, with negative values representing Surface Strategy and positive 
values representing Deep Strategy. The low end of Dimension 2 is represented by items 3,7, 11 
and 15, which are all Surface Motive items. The high end of Dimension 2 is represented by items 
5, 13, and 17, which are Deep Motive items, and item 6, which is a Deep Strategy item. Based on 
the MDS solution, Dimension 1 appears to represent Motive, with negative values representing 
Surface Motive and positive values representing Deep Motive. 



Table 1 

Two-Dimensional MDS Solution 



Item # 


Dimension 1 


Dimension 2 


1 


0.088 


0.736 


2 


1.281 


0.266 


3 


0.733 


-2.071 


4 


-1.230 


-0.029 


5 


0.548 


1.017 


6 


0.161 


1.849 


7 


0.031 


-2.154 


8 


-1.656 


0.538 


9 


0.686 


0.187 


10 


1.291 


0.145 


11 


1.547 


-1.300 


12 


-1.296 


0.283 


13 


-0.685 


1.075 


14 


1.015 


0.085 


15 


1.217 


-1.270 


16 


-1.572 


0.306 


17 


0.027 


1.279 


18 


-0.879 


-0.759 


19 


0.207 


-0.145 


20 


-0.611 


-0.530 


21 


0.543 


0.685 


22 


-1.444 


-0.194 





Dimension 1 

Figure 3. Configuration obtained by applying multidimensional scaling to the R-LPQ-2F. The dimensions were 
scaled (i.e., normalized) to a root-mean-square value of 1, and the dimension coefficients were adjusted to 
compensate. Dimension 1 appears to reflect “Strategy” and Dimension 2 appears to reflect “Motive.” 







Although items do not need to neatly map onto the factors in the hierarchical factor 
model (Davison & Skay, 1991), most of the R-LPQ-2F items in our study did. This is further 
evidence of the construct validity of the latent model. Those that did not were items 6, 1 1, and 
15. Item 6 is “I like constructing theories to fit odd things together.” The content of this question 
can be interpreted as either a Strategy or a Motive, explaining a possible reason why it fell on the 
Motive dimension instead of Strategy. Items 11 and 15 do fall under Surface Motive, but seem to 
also fall in the MDS solution under Deep Strategy. These items have to do with doing well in 
school in order to get a better job, so we do not know why they fall on both dimensions. 
Although it is not the purpose of the current study to improve the R-LPQ-2F, it is possible that 
removing items 6, 1 1, and 15 or changing the latent model to compensate for these items might 
improve the model fit obtained from our factor analysis. 

Profile Analysis 

MDS cannot describe individual differences until the profile analysis is applied. PAMS 
can be used to build profiles (i.e., interpret the MDS dimensions as latent profiles) through re- 
parameterizing the linear latent variable model. These profiles can be used diagnostically 
because PAMS can be used to determine group membership of people where the membership is 
not known in advance of the analysis. A PAMS model calculates person parameters which are 
essentially profile match indices that signify the direction and magnitude of the match between 
the actual profile of the person and the dimension profile. Factor analysis emphasizes how tasks 
vary in their sensitivity to person variates whereas MDS emphasizes how people vary in their 
sensitivity to the task dimensions. A PAMS model therefore studies the latent “person,” that is 
“types” among people as opposed to “factors” among variables (Ding, 2001), so that the latent 
variables can be interpreted as profile patterns (Ding, 2006). This is essentially a variation of the 




traditional factor model. PAMS studies clusters of people and each cluster is a hypothetical 
“prototypical person.” Person parameters (i.e., dimension weights) are derived by linearly 
regressing each person’s observed scores onto the dimension scale values obtained from the 
MDS analysis (Davison, Kim, & Ding, 2001). 

A fit statistic is derived in this analysis (i.e., the R" from the regression) indicating the 
proportion of variance in an individual’s observed data that can be accounted for by the profiles 
(Davison et al., 2001; Ding, 2006). This fit statistic is important to identify individuals who 
develop in an idiographic manner or answered the instrument randomly and therefore do not fit 
within the overall model. It can also represent the credibility of an individual’s response as some 
may over- or under-exaggerate responses or not take the assessment seriously. It can be used to 
calculate the F-statistic and probability value used in regression to determine whether any of the 
explanatory variables are statistically related to the dependent variable. This probability value 
represents whether an individual fits within the overall model. PAMS goes beyond factor 
analysis and allows a researcher to look at whether an individual can be accurately described in 
the context of the latent model, such as is the case with unreliable respondents. 

We computed person parameters and fit statistics for everyone in our sample. Table 2 
contains the person parameters (i.e., dimension weights), level parameter (i.e., intercept from the 
regression), fit and significance, cumulative GPA, and semester GPA at the university for eight 
students from our sample (see Figure 4 for a plot of the person parameters). These eight students 
are good illustrations of how PAMS can be used to assign profiles for diagnostic use. Student 
198 is an example of one who has poor fit and poor probability of being accurately placed in a 
profile. Since the R-LPQ-2F has been shown to have good reliability and construct validity, and 
since the MDS solution maps fairly close to the hierarchical factor model, this student most 




likely responded unreliably. It might be worthwhile to re-administer the R-LPQ-2F or collect 



data from other sources for this student. 

The other students significantly fit within the overall model and can be assigned to a 
profile. Students 5 and 275 have positive weights for both dimensions and therefore have both 
Deep Strategy and Deep Motive. Interestingly enough, these students also have the highest 
cumulative GPA of those in the table. Student 275 also has the highest semester GPA (student 5 
took only S/U courses so a GPA could not be calculated). Student 175 is the opposite, having 
both negative weights for both dimensions and therefore have both Surface Strategy and Surface 
Motive. This student has the lowest cumulative GPA of those in the table and one of the lowest 
semester GPAs. Students 23, 81, and 99 have Deep Strategy and Surface Motive and student 58 
has Surface Strategy and Deep Motive. 



Table 2 



PAMS Person Parameters for Eight Students: R-LPQ-2F Profiles 



Student 


Dimension 1 Wt. 
(Strategy) 


Dimension 2 Wt. 
(Motive) 


Level Par. 


Fit (R 2 ) 


p-value 


Cumulative 

GPA 


Semester 

GPA 


5 


1.036 


0.267 


3.136 


.536 


.004 


4.000 


S 


23 


0.358 


-0.729 


2.818 


.503 


.006 


3.289 


2.750 


58 


-0.854 


0.145 


1.864 


.432 


.015 


2.726 


2.879 


81 


0.543 


-0.444 


2.000 


.358 


.033 


3.271 


2.685 


99 


0.734 


-0.534 


3.227 


.414 


.018 


3.750 


4.000 


175 


-0.116 


-0.748 


3.000 


.456 


.011 


1.629 


2.018 


198 


0.090 


-0.031 


3.136 


.004 


.840 


2.119 


1.686 


275 


1.358 


0.432 


2.909 


.905 


<.001 


4.000 


4.000 




Figure 4. Profile Plot for Eight Students 




Figure 4. Configuration obtained by plotting the dimension weights for each respondent. 



PAMS describes people in terms of continuous person profile indices that specify to what 
extent people are mixtures of the various types whereas factor analysis describes people in terms 
of discrete groupings. PAMS profile information can be utilized clinically to make differential 
diagnoses and to design appropriate interventions based on an individual’s profile pattern. To 
illustrate this we have also computed the factor scores for the same eight students (see Table 3). 
Since factor analysis describes people in terms of discrete groupings, the dimensions of Motive 
and Strategy are represented as the factors Surface Motive, Deep Motive, Surface Strategy, and 
Deep Strategy. Students 81, 99, 175, and 198 have almost identical factor scores for Surface 
Motive and Deep Motive and students 81 and 99 also have almost identical factor scores for 




Surface Strategy and Deep Strategy. These students cannot be profiled accurately using factor 
scores. The PAMS model can classify each one though, as seen above in Table 2. 

Table 3 



Factor Scores for Eight Students 



Student 


Surface Motive 


Deep Motive 


Surface Strategy 


Deep Strategy 


5 


13 


26 


12 


18 


23 


18 


13 


17 


14 


58 


4 


8 


22 


7 


81 


13 


13 


9 


9 


99 


20 


19 


16 


16 


175 


17 


17 


22 


10 


198 


14 


13 


23 


19 


275 


13 


25 


8 


18 



Ecological Validity 

We have discussed how PAMS can be used to assign individuals to profiles based on a 
latent model. The next step is to determine whether these profiles have a relationship with 
behavioral measures such as semester GPA and whether or not the student is retained the 
following semester. To do this we conducted a multiple regression, regressing each student’s 
semester GPA onto their level and dimension parameters. For retention we conducted a logistic 
regression. We included gender (i.e., male or female), race (i.e., white or non-white), credit type 
(i.e., resident credit or distance education), and class level (i.e., graduate or undergraduate) as 
controls in the models. Of the 329 student volunteers, 29 had missing values for one or more of 
the variables, leaving us with 300 for the regressions. The results for the semester GPA model 
show that Strategy has a significant positive relationship with academic performance (see Table 
4). This intuitively makes sense since surface strategies revolve around learning for 
examinations, memorization, and devoting as little time to learning as possible and deep 
strategies revolve around learning to understand the material and relating ideas. Motive was also 
significant, but had a negative relationship with academic performance. This makes sense since 



Deep Motive involves having an intrinsic interest and commitment whereas Surface Motive has 




to do more with doing well in school and not wanting to get poor marks. These results indicate 
that Strategy and Motive are significantly related to academic performance. 

Since PAMS allows a researcher to identify students who develop in an idiographic 
manner and are not explained by the model through the calculation of a fit statistic, we decided 
to run the regression again, only this time removing students with a low fit. This capability is not 
inherent in factor scores and is an important component of a PAMS analysis. Students were 
removed based on the probability of their fit (using a = .05), such as student 198. Doing this 
allows us to look more closely at the relationship of each profile to semester GPA, which is 
important because only those who fit within the model can be profiled and helped with the use of 
an intervention. After removing those with poor fit we were left with 244 for the regression 
analysis. Overall the results are exactly the same as when all students were left in (see Table 4). 
This second regression is noteworthy, though, because it shows that the PAMS method can be 
applied from a diagnostician viewpoint in order to make more accurate decisions and determine 
which students we may need additional data for. 

Table 4 

Regression Parameters for Regressing Semester GPA onto PAMS Dimensions 

All Observations Excluding Observations 



Variable 


Parameter 

Estimate 


Standardized 

Estimate 


Parameter 

Estimate 


Standardized 

Estimate 


Intercept 


3.1594*** 


0 


2.9755*** 


0 


Gender ( 1 = Female, 0 = Male) 


-0.0609 


-0.0310 


-0.1903 


-0.0983 


Race (1 = White, 0 = Non-white) 


0.3263* 


0.1097 


0.2797 


0.0970 


Credit Type (1 = Resident Credit, 0 = Distance Education) 


-0.1197 


-0.0443 


-0.1320 


-0.0526 


Class Level (1 = Graduate, 0 = Undergraduate) 


0.4092*** 


0.1987 


0.3873*** 


0.1992 


Dimension 1 Wt. (Strategy) 


0.4963*** 


0.2102 


0.6677*** 


0.2538 


Dimension 2 Wt. (Motive) 


-0.3242** 


-0.1485 


-0.3774** 


-0.1866 


PAMS Level 


-0.2144 


-0.0849 


-0.1578 


-0.0644 



Note. The model using all observations has an adjusted R =.0749 and the model excluding observations has an 
adjusted R 2 =.0825. 

*p< .10. ** p < .05. *** p < .01. 



As with most diagnostic tests and self-reports, some students are going to give invalid 
results. The ability to eliminate unreliable respondents is not a new idea, as it is common in such 




instruments as the MMPI. Hahn (2005) notes that when “using a client’s self-report, it is crucial 



to determine the credibility of the individual’s performance — for example, whether he or she has 
cooperated fully with the evaluation” (p. 65). The advantage with PAMS is that this can be done 
without the need for an extra validity scale, as the fit statistic in PAMS is a measure of how well 
an individual’s item responses fit within the model. This ability allows a clinician to apply the 
appropriate assessment and offer more relevant assistance to students. 

We also ran a multiple regression, regressing each student’s semester GPA on the factor 
scores, in order to show a comparison between PAMS and factor analysis. In this model none of 
the subscales were significant, showing no evidence of ecological validity (see Table 5). Also, 
whereas the PAMS model has only two dimensions as independent variables, the factor analysis 
model has four subscales, thus increasing the complexity of drawing inference. It is possible that 
the complexity of interpreting each side of each dimension as a discrete factor instead of 
continuous dimensions is why this model has no significance. It makes sense that Deep Strategy 
is related to Surface Strategy and Deep Motive is related to Surface Motive, and that looking at 
each separately without including that relationship could affect the significance of each in the 
model. 



Table 5 



Regression Parameters for Regressing Semester GPA onto Factor Scores 



Variable 


Parameter Estimate 


Standardized Estimate 


Intercept 


3.0935*** 


0 


Gender ( 1 = Female, 0 = Male) 


0.0025 


0.0013 


Race (1 = White, 0 = Non-white) 


0.3137* 


0.1055 


Credit Type (1 = Resident Credit, 0 = Distance Education) 


-0.1216 


-0.0450 


Class Level (1 = Graduate, 0 = Undergraduate) 


0.4267*** 


0.2072 


Deep Motive Subscale 


-0.0168 


-0.0825 


Deep Strategy Subscale 


0.0139 


0.0454 


Surface Motive Subscale 


0.0162 


0.0584 


Surface Strategy Subscale 


-0.0211 


-0.0951 



• • 7 ~r- 

Note. This model has an adjusted R“=.05 1 8. 
*p < .10. ** p < .05. *** p < .01. 




Next we conducted logistic regressions, regressing retention on the PAMS dimensions 



and regressing retention on the factor scores. Retention was coded as 1 for students who enrolled 
in the next semester and 0 for those who did not. Those students who graduated prior to the next 
semester were removed prior to the analysis. Like before, we ran this regression including all 
observations and again excluding those with low fit (see Table 6). The model with all 
observations did not yield a significant relationship for Strategy and Motive. Strategy did have a 
mildly significant negative relationship with retention in the model that removed those with poor 
fit though. This indicates that those who use more surface strategies are for some reason more 
likely to enroll again the next semester. This makes sense since Deep Strategy deals with trying 
to understand the material and relate it with other subjects and Surface Strategy deals with 
studying for examinations and to get by in school. Those utilizing deep strategies might be 
transferring out or may be leaving because they are not concerned with learning solely to do well 
in school. Finally, we conducted this logistic regression using the factor subscales instead of the 
PAMS dimensions (see Table 7). We obtained similar results in that Surface Strategy has a 
significant positive relationship with the probability of whether a student will return next 
semester. 



Table 6 



Logistic Regression Parameters for Regressing Retention onto PAMS Dimensions 





All Observations (N = 277) 


Removing Observations 
(N = 227) 


Variable 


Estimate 


Odds Ratio 


Estimate 


Odds Ratio 


Intercept 


-0.5222 


0.593 


0.5496 


1.733 


Gender (1 = Female, 0 = Male) 


-0.1019 


0.903 


-0.0820 


0.921 


Race (1 = White, 0 = Non-white) 


0.4686* 


1.598 


0.5104* 


1.666 


Credit Type (1 = Resident Credit, 0 = Distance Education) 


0.5516*** 


1.736 


0.4581** 


1.581 


Class Level (1 = Graduate, 0 = Undergraduate) 


-0.1018 


0.903 


-0.1391 


0.870 


Dimension 1 Wt. (Strategy) 


-0.9985 


0.368 


-1.5736* 


0.207 


Dimension 2 Wt. (Motive) 


0.4705 


1.601 


0.6218 


1.862 


PAMS Level 


0.8354 


2.306 


0.6734 


1.961 



*p< .10. ** p < .05. *** p < .01. 






Table 7 

Logistic Regression Parameters for Regressing Retention onto Factor Scores (N = 277) 



Variable 


Estimate 


Odds Ratio 


Intercept 


-0.7426 


0.476 


Gender (1 = Female, 0 = Male) 


-0.0871 


0.917 


Race (1 = White, 0 = Non-white) 


0.4768** 


1.611 


Credit Type (1 = Resident Credit, 0 = Distance Education) 


0.5414*** 


1.718 


Class Level (1 = Graduate, 0 = Undergraduate) 


-0.1148 


0.892 


Deep Motive Subscale 


0.0357 


1.036 


Deep Strategy Subscale 


0.0305 


1.031 


Surface Motive Subscale 


-0.0705 


0.932 


Surface Strategy Subscale 


0.1232** 


1.131 



*p< .10. ** p < .05. *** p < .01. 



Discussion 

The LPQ and its subsequent revisions have been part of the development of a conceptual 
understanding of how students learn, their motivation to engage in particular tasks, and the 
strategies they utilize to reach their academic goals. It has been generally accepted throughout 
the literature that these approaches to learning can be reduced to Deep Approach and Surface 
Approach, which encompass the motives and strategies which follow suit. Even with a smaller 
sample, the factor analysis accomplished in this study clearly confirmed the findings seen in 
previous studies as the subscales of Deep Motive, Surface Motive, Deep Strategy, and Surface 
Strategy emerged in the R-LPQ-2F. 

Both factor analysis and PAMS techniques have advantages and disadvantages, and 
depending on the instrument and the latent construct, one may be more appropriate than the other 
for determining how to best use the instrument practically. In the case of the R-LPQ-2F, PAMS 
definitely seems to provide a better understanding of the application of the constructs of 
“motive” and “strategy” to students in higher education. 

Diagnostically, these factors might be utilized to represent personality characteristics, but 
they can be cumbersome and difficult to interpret. That is, in order to use this instrument as a 



means for remediation, student support, or simply reflection, information concerning the student 




needs to be easily attainable and useable. For example, if a student completes the assessment and 
indicates utilization of both deep and surface learning approaches, whether he is more surface 
than deep or more deep than surface will have bearing how that student might alter his learning 
approach. The PAMS model does indeed help look at the individual test taker and a profile that 
represents characteristics of that individual. That is, the PAMS model allows for a “person-level” 
interpretation of the analysis. 

PAMS basically allows for a conversion of the data into profiles so that overarching 
behaviors can be more easily categorized. The analysis forms a representation of how these 
behaviors play out in relationship to one another. A profile is basically a person’s performance 
on a set of scores (Ding, 2001). PAMS extends MDS by interpreting the MDS dimensions as 
latent profiles (i.e., each dimension represents a group of individuals with similar characteristics; 
Ding, 2001, 2006; Kim et al., 2007). PAMS represents what profiles of variables exist in the 
population and how individuals differ in those profiles (Ding, 2006). In this particular case, the 
dimensions Motive and Strategy and level of processing (deep and surface) allow for a learner 
personality “type” to be developed. More specifically, the dimensions show how a person 
functions in the academic environment which can then be tied to other variables. 

In this study, the dimension of Strategy can clearly be seen as a continuum from surface 
to deep. In addition, student profiles which fall along that dimension have been shown to relate 
significantly to GPA. In short, the capacity to utilize this instrument diagnostically has become 
readily apparent. A student profile is directly linked to the desired or undesired behaviors which 
now can be addressed, altered, and remediated. Although it seems intuitively obvious for 
students to use deep strategies because they are more efficacious, some students are unable to 
assess their own strategy use, and they continue to use methods that do not work. These students 




may be unable to see that their techniques are unsuccessful, or they may simply have a limited 
repertoire of study skills from which to pull. Helping students become more aware of the 
strategies they use, helping them monitor and regulate these strategies, and helping them choose 
between more successful and less successful strategies is essential. Counselors and advisors 
would be able to use these student profiles to determine whether it is indeed the strategies 
employed by the student that are affecting the academic outcomes. Also inherent in PAMS is the 
ability to identify individuals who do not fit within the model or who respond unreliably which 
allows one to identify those individuals with which further data, possibly from other outside 
sources, are needed. This might indicate the need for more diagnostic information or further 
testing in other areas and allows for the maximization of resources for helping individual 
students. 

The dimension of Motive can also clearly be seen as a continuum from surface to deep. 
Student profiles which fall along that dimension have been shown to relate negatively to GPA. 
This makes sense since Deep Motive involves having an intrinsic interest and commitment 
whereas Surface Motive has to do more with doing well in school and not wanting to get poor 
marks. Unfortunately, this is less than ideal. Our mission is to develop students into lifelong 
learners and yet get good grades. We may not want to intervene with students based on Motive, 
but instead intervene with our curriculums to change the nature of Motive’s relationship with 
academic performance. 

As the current research offers a new perspective on the learning approach dimensions, 
allows the development of profiles, and affords an opportunity to diagnostically assess students 
concerning their strategies and motives, there is still the question of what other academic and 
personal factors are related to both dimensions. We have clearly identified the continuums of 




Strategy and Motive, surface to deep, and have demonstrated those characteristics to be strongly 
related to semester GPA and mildly related to retention. For future research it might be beneficial 
to collect drop/withdraw/failure statistics, learning outcomes, variables related to what the 
students do after they graduate, and how possible interventions and curriculums improve 
students’ strategies and motives. 
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Appendix A 



Revised Learning Process Questionnaire (R-LPQ-2F) 

© John Biggs and David Kember 

This questionnaire has a number of questions about your attitudes towards your studies and your 
usual way of studying. 

There is no right way of studying. It depends on what suits your own style and the course you are 
studying. It is accordingly important that you answer each question as honestly as you can. If you 
think your answer to a question would depend on the subject being studied, give the answer that 
would apply to the subject(s) most important to you. 

Please fill in the appropriate circle alongside the question number on the “General Purpose 
Survey/ Answer Sheet’’. The letters alongside each number stand for the following response. 

A — this item is never or only rarely true of me 
B — this item is sometimes true of me 
C — this item is true of me about half the time 
D — this item is frequently true of me 
E — this item is always or almost always true of me 

Please choose the one most appropriate response to each question. Fill the oval on the Answer 
Sheet that best fits your immediate reaction. Do not spend a long time on each item: your first 
reaction is probably the best one. Please answer each item. 

Do not worry about projecting a good image. Your answers are CONFIDENTIAL. 

Thank you for your cooperation. 

(1) 1 find that at times studying makes me feel really happy and satisfied. 

(2) I try to relate what I have learned in one subject to what I learn in other subjects. 

(3) I am discouraged by a poor mark on a test and worry about how I will do on the next test. 

(4) I see no point in learning material which is not likely to be in the examination. 

(5) I feel that nearly any topic can be highly interesting once I get into it. 

(6) I like constructing theories to fit odd things together. 

(7) Even when I have studied hard for a test, I worry that I may not be able to do well in it. 

(8) As long as I feel I am doing enough to pass, I devote as little time to studying as I can. There 
are many more interesting things to do. 

(9) I work hard at my studies because I find the material interesting. 

(10) I try to relate new material, as I am reading it, to what I already know on that topic. 

(11) Whether I like it or not, I can see that doing well in school is a good way to get a well-paid 
job. 

(12) I generally restrict my study to what is specifically set as I think it is unnecessary to do 
anything extra. 




(13) I spend a lot of my free time finding out more about interesting topics which have been 
discussed in different classes. 

(14) When I read a textbook, I try to understand what the author means. 

(15) I intend to get my A Levels [or equivalent qualification] because I feel that I will then be 
able to get a better job. 

(16) I find it is not helpful to study topics in depth. You don’t really need to know much in order 
to get by in most topics. 

(17) I come to most classes with questions in mind that I want answering. 

(18) I leam some things by rote, going over and over them until I know them by heart even if I 
do not understand them. 

(19) I find I am continually going over my school work in my mind at times like when I am on 
the bus, walking, or lying in bed, and so on. 

(20) I find the best way to pass examinations is to try to remember answers to likely questions. 

(21) 1 like to do enough work on a topic so that I can form my own conclusions before I am 
satisfied. 

(22) I find I can get by in most assessment by memorising key sections rather than trying to 
understand them. 




Appendix B 



Scales in the Revised Learning Process Questionnaire (R-LPQ-2F) 

© John Biggs and David Kember 

The number in parentheses is the item number in the questionnaire. 

Deep approach 
Deep motive 

Intrinsic interest 

I find that at times studying makes me feel really happy and satisfied. (1) 

I feel that nearly any topic can be highly interesting once I get into it. (5) 

I work hard at my studies because I find the material interesting. (9) 

Commitment to work 

I spend a lot of my free time finding out more about interesting topics which have been discussed 
in different classes. (13) 

I come to most classes with questions in mind that I want answering. (17) 

I find I am continually going over my school work in my mind at times like when I am on the 
bus, walking, or lying in bed, and so on. (19) 

I like to do enough work on a topic so that I can form my own conclusions before I am satisfied. 

( 21 ) 

Deep strategy 

Relating ideas 

I try to relate what I have learned in one subject to what I learn in other subjects. (2) 

I like constructing theories to fit odd things together. (6) 

Understanding 

I try to relate new material, as I am reading it, to what I already know on that topic. (10) 

When I read a textbook, I try to understand what the author means. (14) 

Surface approach 

Surface motive 

Fear of failure 

I am discouraged by a poor mark on a test and worry about how I will do on the next test. (3) 
Even when I have studied hard for a test, I worry that I may not be able to do well in it. (7) 




Aim for qualification 

Whether I like it or not, I can see that doing well in school is a good way to get a well-paid job. 
(11) 

I intend to get my A Levels because I feel that I will then be able to get a better job. (15) 

Surface strategy 

Minimizing scope of study 

I see no point in learning material which is not likely to be in the examination. (4) 

As long as I feel I am doing enough to pass, I devote as little time to studying as I can. There are 
many more interesting things to do. (8) 

I generally restrict my study to what is specifically set as I think it is unnecessary to do anything 
extra. (12) 

I find it is not helpful to study topics in depth. You don’t really need to know much in order to 
get by in most topics. (16) 

Memorisation 

I learn some things by rote, going over and over them until I know them by heart. (18) 

I find the best way to pass examinations is to try to remember answers to likely questions. (20) 

I find I can get by in most assessment by memorising key sections rather than trying to 
understand them. (22) 

To calculate scores on the scales use the following response scores. 

A=1,B = 2, C = 3, D = 4, E = 5 

Scores for the two main scales, deep approach (DA) and surface approach (SA), can then be 
calculated by adding the following item scores: 

DA = 1 + 2 + 5 + 6 + 9 + 10 + 13 + 14 + 17 + 19 + 21 
SA = 3 + 4 + 7 + 8 + 1 1 + 12 + 15 + 16 +18 + 20 + 22 

Each contains identifiable strategy (DS and SS) and motive (DM and SM) subscales. The 
subscale and scale scores can be calculated by adding item scores as follows: 

DM = 1 + 5 + 9 + 13 + 17 + 19 + 21 
DS = 2 + 6+ 10 + 14 
SM = 3 +7 + 11 + 15 
SS = 4 + 8 + 12 + 16 +18 + 20 + 22 




